Superlinear parallelisation of the k-nearest neighbor classifier

With m processors available, the k-nearest neighbor classifier can be straightforwardly parallelized with a linear speed increase of factor m. In this paper we introduce two methods that in principle can achieve this aim. The first method splits the test set in m parts, while the other distributes the training set overm sub-classifiers, and merges their m nearest neighbor sets with each classification. For our experiments we use Timbl, an implementation of the k-NN classifier that uses a decision-tree-based data structure for retrieving nearest neigbors. In a range of experiments the first method consistently scales linearly. With the second method we observe cases of both superlinear and sublinear scaling. A high variance in feature weights dampens the effect of the second type of parallelisation, due to the strong weight-based search heuristics already built into Timbl. When feature weights exhibit less variance, superlinear scaling can occur, due to relatively faster nearest neighbor retrieval by sub-classifiers on 1/mth training sets as compared to retrieval by a classifier trained on the full training set.